Deep Neural Network-Based Digital Pre-Distortion for High Baudrate Optical Coherent Transmission

High-symbol-rate coherentoptical transceivers suffer more from the critical responses of transceiver components at high frequency, especially when applying a higher order modulation format. We recently proposed neural network (NN)-based digital pre-distortion (DPD) technique trained to mitigate response 128 GBaud optical coherent transmission system. In this paper, we further detail work and assess NN-based DPD by training it using either direct learning architecture (DLA) or an indirect (ILA), compare performance against Volterra series-based ILA linear DPD. Furthermore, deliberately increase transmitter nonlinearity three DPDs schemes. The DLA performs best among contenders. comparison DPD, provides than 1 dB signal-to-noise ratio (SNR) gains output conventional receiver DSP for uniform 64-quadrature amplitude (QAM) PCS-256-QAM signals. Finally, enables achieving record 1.61 Tb/s net rate on single channel after 80 km standard mode fiber (SSMF).